feat(playground): accumulate errors while parsing span attributes #4941
Conversation
dosubot
bot
added
the
size:XL
| import { PlaygroundSpan } from "./spanPlaygroundPageLoader"; | ||
|
|
||
| /** | ||
| * Checks if a string is a valid chat message role | ||
| */ | ||
| export function isChatMessageRole(role: unknown): role is ChatMessageRole { | ||
| return chatMessageRoles.includes(role as ChatMessageRole); | ||
| return chatMessageRolesSchema.safeParse(role).success; |
There was a problem hiding this comment.
we don't have to use this but i wanted to add it to the shcmea for ChatMessages so figured would update here as well
| * The role of a chat message with a LLM | ||
| */ | ||
| export type ChatMessageRole = (typeof chatMessageRoles)[number]; | ||
| export enum ChatMessageRole { |
There was a problem hiding this comment.
needed these actual values in a few places so switched to enum
| import { PlaygroundInstanceProps } from "./types"; | ||
|
|
||
| interface PlaygroundOutputProps extends PlaygroundInstanceProps {} | ||
|
|
||
| export function PlaygroundOutputMessage({ message }: { message: ChatMessage }) { |
There was a problem hiding this comment.
not sure if we want to show this card in all cases but wanted it for span parsing, seems to be a common way to display
There was a problem hiding this comment.
I wouldn't export.
What do you mean you wanted it for "span parsing" ?
There was a problem hiding this comment.
oh yeah export was a mistake
also the parsing wording was a brain fart haha, wanted it for displaying output messages parsed from the original span** is what i meant to say
| message={{ | ||
| id: generateMessageId(), | ||
| content: output, | ||
| role: ChatMessageRole.ai, | ||
| }} |
There was a problem hiding this comment.
can update output messages to not need id, seems fine to keep for now
| // expect(getChatRole("bot")).toEqual("ai"); | ||
| // expect(getChatRole("system")).toEqual("system"); | ||
| // expect(getChatRole("human:")).toEqual("user"); | ||
| expect(getChatRole("bot")).toEqual("ai"); |
| @@ -28,6 +29,7 @@ export function PlaygroundInstance(props: PlaygroundInstanceProps) { | |||
| return ( | |||
| <PanelGroup direction={isSingleInstance ? "horizontal" : "vertical"}> | |||
| <Panel defaultSize={50} order={1} css={panelContentCSS}> | |||
| <PlaygroundInstanceErrors {...props} /> | |||
There was a problem hiding this comment.
I think we eventually put this in a banner as a single "warning". Can always refactor
| <Flex direction="column" gap={"size-50"}> | ||
| {parsingErrors.map((error, i) => { | ||
| return ( | ||
| <Alert | ||
| variant="warning" | ||
| dismissable | ||
| key={error} | ||
| onDismissClick={() => { | ||
| updateInstance({ | ||
| instanceId, | ||
| patch: { | ||
| parsingErrors: parsingErrors.filter((_, j) => i !== j), | ||
| }, | ||
| }); | ||
| }} | ||
| > | ||
| {error} | ||
| </Alert> | ||
| ); | ||
| })} | ||
| </Flex> |
There was a problem hiding this comment.
Maybe one alert with a detail element might be cleaner.
There was a problem hiding this comment.
yeah i like that, i can refactor
| import { PlaygroundInstanceProps } from "./types"; | ||
|
|
||
| interface PlaygroundOutputProps extends PlaygroundInstanceProps {} | ||
|
|
||
| export function PlaygroundOutputMessage({ message }: { message: ChatMessage }) { |
There was a problem hiding this comment.
I wouldn't export.
What do you mean you wanted it for "span parsing" ?
| if (isChatMessages(instance.output)) { | ||
| const messages = instance.output; | ||
|
|
||
| return messages.map((message, index) => { | ||
| return <PlaygroundOutputMessage key={index} message={message} />; | ||
| }); | ||
| } |
There was a problem hiding this comment.
Oh as in this was the original output?
There was a problem hiding this comment.
yes, displaying the original output but then also displaying new outputs in the same way
Parker-Stafford
force-pushed
the
parker/4922-accumulate-errors-while-parsing
branch
from
f918d83
to
691beb7
Compare
mikeldking
force-pushed
the
playground
branch
from
cf18e34
to
5990585
Compare
Parker-Stafford
force-pushed
the
parker/4922-accumulate-errors-while-parsing
branch
from
319d479
to
4efc271
Compare
dosubot
bot
added
size:XXL
Parker-Stafford
force-pushed
the
parker/4922-accumulate-errors-while-parsing
branch
from
4efc271
to
02e1d82
Compare
dosubot
bot
added
size:XL
Parker-Stafford
force-pushed
the
parker/4922-accumulate-errors-while-parsing
branch
from
ee38d2e
to
6f79a05
Compare
Parker-Stafford
deleted the
parker/4922-accumulate-errors-while-parsing
branch
) * feat(playground): accumulate errors while parsing span attributes * update tests to include message id's / tools * update output for runs * update comments / test names * move parsing errors into span playground banners * cleanup spanplaygroundpage * add function to verify zod schema matches type
resolves #4922
Errors
Output